Focused on the issue that developers cannot quickly figure out the models they need from various models, an automatic annotation method of visual deep neural network based on natural language processing technology was proposed. Firstly, the field categories of visual neural networks were divided, the keywords and corresponding weights were calculated according to the word frequency and other information. Secondly, a keyword extractor was established to extract keywords from paper abstracts. Finally, the similarities between extracted keywords and the known weights were calculated in order to obtain the application fields of a specific model. With experimental data derived from the papers published in three top international conferences of computer vision: IEEE International Conference on Computer Vision(ICCV), IEEE Conference on Computer Vision and Pattern Recognition(CVPR) and European Conference on Computer Vision(ECCV), the experiments were carried out. The experimental results indicate that the proposed method provides highly accurate classification results with a macro average value of 0.89. The validity of this proposed method is verified.